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Abstract 


This  report  summarizes  the  results  of  research  performed  under 
Grant  AF0SR-74-2601  for  the  period  September  1,  1975  to  August  31,  1976. 
The  research  was  concerned  with  problems  in  communications  theory  and 

information  theory  and  their  applications.  Specific  problems  considered 

/ 

were: 

(a)  Soft  decision  decoding  of  linear  block  codes. 

(b)  Application  of  coding  to  computers. 

(c)  Coding  for  noisy  channels  including  multi-user  channels. 
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I . Introduction 

This  is  an  interim  technical  report  under  Grant  AF0SR-74-2601 
sponsored  by  the  Air  Force  Office  of  Scientific  Research  for  the  period 
September  1 , 1975  to  August  31  , 1976.  Two  previous  interim  reports,  one 
issued  in  September  1974,  and  the  other  issued  in  October  1975  covered 

J 

the  work  conducted  during  the  first  two  years  of  this  grant,  September  1,  1973 
to  August  31,  1975.  This  work  was  conducted  at  the  University  of 
Massachusetts,  Amherst,  Massachusetts.  It  is  a follow-on  to  research  pre- 
viously conducted  at  the  Polytechnic  Institute  of  Brooklyn. 

The  research  performed  under  this  contract  was  primarily  concerned 
with  problems  in  cornnunications  theory  and  information  theory.  Specific 
problems  considered  were  soft  decision  decoding  for  linear  block  codes, 
applications  of  coding  to  computers,  and  coding  for  noisy  channels  in- 
cluding multi-user  channels. 

In  Section  II,  a brief  summary  of  the  research  activity  for  the  grant 
period  is  presented.  The  details  of  this  research  are  contained  in  the 
published  papers. 

Section  III  contains  a list  of  such  papers  plus  symposia  and  in- 
vited talks  concerned  with  research  conducted  under  this  grant.  Section 
IV  mentions  an  award  presented  to  the  principal  investigator  for  a paper 
on  previous  research  on  multi-user  communications.  Finally  Sections  V 
and  VI  contain  titles  of  dissertations  and  journal  articles  on  research 
supported  by  previous  AFOSR  contracts  and  grants. 
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1 1 . Summary  of  Research 

1.  Soft  Decision  Decoding  for  Linear  Block  Codes 

A new  method  has  been  found  for  achieving  maximum  likelihood  detection 
of  the  qk  code  words  in  a (n,k)  linear  block  code  with  symbols  from  GF(q). 

This  method  can  utilize  soft  decisions  (i.e.  analog  signals)  as  well  as 
hard  decisions.  For  an  additive  channel  where  the  noise  in  each  signalling 
interval  is  independent  of  all  other  signalling  intervals,  the  complexity 
of  the  decoder  when  using  either  soft  or  hard  decisions  is  proportional  to 
qn-k  (or  sometimes  q'  where  e.  < n-k).  The  method  is  analogous  to  the  use 
of  the  Viterbi  algori thm^ 1 ^for  convolutional  codes. 

To  compare  this  method  with  conventional  word  decoding,  consider  a 
(31,26)  binary  code.  Word  correlation  requires  comparing  the  received 

) 

waveform  with  2^&  distinct  code  words.  The  method  referred  to  above  re- 
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quired  storing  no  more  than  2 quantities  at  any  time. 

Rather  than  explain  the  method  in  general,  we  consider  here  only  a 
cyclic  (15,11)  binary  Hamming  code.  The  details  of  the  generalization  to 
nonbinary,  noncyclic,  codes  is  covered  in  a paper  which  has  been  prepared 
on  this  subject. 

Example  Decoding  a (15,11)  Binary  Cyclic  Code 

Consider  the  (15,11)  binary  cyclic  code  with  generator  polynomial 
g ( x ) = x4  + x + 1 . One  method  of  forming  the  code  words  for  this  code 
is  via  the  encoding  circuitry  shown  in  Figure  1. 


r 
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^ tWpt/T 

Figure  1.  Encoder  for  code  with  n=15,  k=ll,  and  r=4 


For  the  first  11  clock  pul  s,  switches  S1  and  S2  are  in  position  1 and 
the  input  consists  of  the  11  message  digits.  For  the  next  4 clock  pulses, 
switches  S1  and  S2  are  in  position  2 and  zeros  are  fed  in  at  the  input. 

The  first  11  digits  which  appear  at  the  output  are  then  the  11  message 
digits  and  the  last  4 digits  which  appear  at  the  output  are  the  4 check 
digits.  As  an  example  we  show  the  operation  of  this  circuit  if  the  message 
digits  are  10100101110. 
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State  of  Encoder 


input  1st  flip  flop  2nd  flip  flop  3rd  flip  flop  4th  flip  flop  output 


The  state  sequence  for  the  encoder  for  this  Input  Is  then 


0000-1100+0110+1111-1011-1001-0100-0010 
llOl-frOllO-tllll+IOOl+OIOl^OOIO  + OOOI+OOOO 
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We  now  form  what  Is  known  as  the  trellis  diagram  for  this  code.  Prior 
to  a message  digit  entering  the  encoder,  the  encoder  contains  the  digits 
0 0 0 0 In  Its  four  flip  flops.  We  say  the  encoder  Is  in  0 0 0 0 state. 

If  a 0 enters  the  encoder  as  the  message  digit,  the  encoder  remains  in  the 
0000  state  while  If  a 1 enters  the  encoder,  the  encoder  goes  to  the 
1100  state.  This  Is  shown  In  Figure  2.  The  possible  states  after 

0 0 0 0 v— 0 0 0 0 

"^1  1 o 0 

Figure  2.  Start  of  Trellis  for  Encoder  of  Figure  1. 

2,  3 and  4 message  digits  are  shown  In  Figure  3.  Note  that  after  4 message 
digits  all  16  possible  states  are  reached  by  each  of  the  16  possible  mes- 
sage sequences.  For  the  next  7 message  digits  the  trellis  does  a peculiar 
thing.  Each  state  is  entered  from  exactly  two  other  states.  The  trellis 
for  these  next  7 message  digits  Is  shown  in  Figure  4.  Finally  on  the  12th 
through  15th  clock  pulse,  zeros  are  entered  Into  the  encoder  and  the  trellis 
collapses  as  shown  in  Figure  5.  The  trellis  for  the  complete  encoder  thus 
consists  of  a concatenation  of  Figure  3,  7 sections  of  Figure  4 and  the 
end  of  the  trellis  as  in  Figure  5.  In  every  case  the  branches  of  the  trellis 
are  labeled  by  0 or  1 corresponding  to  the  digit  of  the  code  word  trans- 
mitted. A code  word  Is  then  Identified  by  a path  throuqh  the  trellis  from 
beginning  to  end  where  each  code  word  Is  Identified  by  a unique  path.  A 
typical  path  Is  shown  In  Figure  6 corresponding  to  the  code  word 
10100101110100  1. 


0 0 0 0 
0 0 0 1 
0 0 10 
0 0 11 
0 10  0 
0 10  1 
0 110 
0 111 
10  0 0 
10  0 1 
10  10 
10  11 
110  0 
110  1 
1110 
1111 


0 0 0 0 
0 0 0 1 
0 0 10 
0 0 11 
0 10  0 
0 10  1 
0 110 
0 111 
10  0 0 
10  0 1 
10  10 
10  11 
110  0 
110  1 
1110 
1111 


Figure  3.  Trellis  for  up  to  4 message  digits 


Figure  4.  Trellis  structure  for  message  digits  5 through  11 
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0 0 0 0 
0 0 0 1 
0 0 10 
0 0 11 
0 10  0 


..  CL 

I 


. o ' 


0 10  1 
0 110 
0 111 
10  0 0 
10  0 1 
10  10 
10  11 
110  0 
110  1 
1110 
1111 
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Figure  5.  End  of  Trellis  for  clock  pulses  12  through  15 


0 0 0 0 
0 0 0 1 
0 0 10 
0 0 11 
0 10  0 
0 10  1 
0 110 
0 111 
10  0 0 
10  0 1 
10  10 
10  11 
110  0 
110  1 
1110 
1111 


Figure  6.  Path  in  Trellis  for  Code  Word  101001011101^01 
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Once  the  trellis  has  been  constructed  for  the  code,  the  Viterbi^ 
algorithm  can  be  used  to  find  the  maximum  likelihood  path  through  the 
trellis.  This  algorithm  can  be  used  for  either  a hard  decision  receiver 
or  a soft  decision  receiver.  (A  soft  decision  receiver  is  one  where 
channel  measurement  information  is  available  at  the  decoder.)  However, 
since  we  are  concerned  with  a code  where  a very  simple  al gebraic  de- 
coding algorithm  is  known  for  hard  decisions,  it  would  appear  that  the 
technique  discussed  here  would  only  be  of  interest  in  the  case  of  soft 
decisions. 

The  use  of  this  technique  has  been  suggested  for  decoding  constant 
weight  binary  codes  in  a particular  fading  channel.  (This  is  research 
presently  being  performed  by  Stein  Associates  under  Contract 
N-00140-76-C-6533  for  the  Naval  Underwater  Systems  Center.)  The  details 
of  this  work  are  not  included  here,  but  rather  a pair  of  curves  are  shown 
in  Figure  7,  giving  the  bit  error  probability  versus  average  ratio  of  the 
energy  per  bit  to  noise  power  density  for  two  competing  systems.  The  curve 
labeled  "Single  H(20,5)“  is  a coded  system  previously  considered  by  others. 
The  curve  labeled  "Concatenated:  [9,8]"  is  a system  which  uses  the  tech- 
niques developed  under  this  grant.  The  system  has  2 ^ code  words  and  it 
would  have  been  impossible  to  consider  such  a system  if  one  had  to  con- 
sider comparing  the  received  signal  with  each  of  the  code  words. 


2.  Application  of  Coding  to  Computers 

A particular  application  of  the  Chinese  Remainder  Theorem  to  the 
design  of  fault  tolerant  computers  has  been  investigated.  A brief  summary 
of  this  work  follows. 

The  basic  theorem  to  be  used  is  the  following:  Let  m^ , be 

L positive  integers  that  are  relatively  prime  in  pairs.  Let  “I"  be  any 

L 

non-negative  integer  less  than  m = n m. . Then  "I"  can  be  uniquely  recon- 

i=l  1 

structed  from  its  remainders,  r^ , r2,---,rL  where  I = m^  + r^ 

0 £ ri  < mi » i = 1 ,2,- • • ,L. 

An  example  is  given  in  the  following  table  for  m-j  = 2,  m£  = 3 and 


An  important  corollary  to  this  theorem  is: 

Let  S be  a subset  of  the  integers  , m2,-*-mL.  If  I is  an  integer 

in  the  range  0 <_  I < n m. , then  I can  be  uniquely  reconstructed  from  the 

i=S  1 

remainders  corresponding  to  the  m^  in  this  subset. 

i 
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To  see  that  this  is  the  case  consider  the  previous  example  where 
= 2,  m2  = 3,  m3  = 5.  Then  we  can  consider  three  sets  S as  follows: 


1 
0 
1 

2 

3 

4 

5 


S1  = {m^  ,ni2  i 


n 

0 

1 

0 

1 

0 

1 


2 1 


0 

1 

2 

0 

1 

2 


1 

0 

1 

2 

3 

4 

5 


{ m-j  ,11)3} 

rl  r: 


0 

1 

0 

1 

0 

1 


0 

1 

2 

3 

4 
0 


1 
0 
1 

2 

3 

4 

5 


S3  = {1112,1113} 

r2 
0 

1 
2 
0 
1 
2 


r3 

0 

1 

2 

3 

4 
0 
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Thus  given  the  remainders  (r^,  r2,  r3),  any  two  of  these  remainders  can 

uniquely  determine  an  integer  I in  the  range  0 £ I < 5. 

Finally  we  consider  a second  corollary  to  the  Chinese  Remainder  Theorem: 

Let  I be  a non-negative  integer  in  the  range  0 <_  I < M.  Let 

m-,  < mo  < •••  < m be  positive  integers  that  are  relatively  prime  in  pairs. 

1 N s 

Let  s be  the  smallest  integer  such  that  n nu  >_  M.  Then  "I"  can  be 

i=l 

uniquely  determined  from  any  s remainders  from  the  set  {r^ ,r2, • • • ,rN) . 

Let  s and  N be  defined  as  in  the  previous  corollary.  Consider  the 
set  of  remainders  where  now  F of  these  remainders  are  erased 

(i.e.  are  missing)  and  T of  them  are  in  error.  Assume  that  2T  + F <_  N - s. 
Then  one  can  uniquely  determine  I from  the  remaining  N-F  unerased  remainders, 
T of  which  are  in  error. 
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As  an  example,  let 

m^  = 97,  m2  = 101,  m-j  = 103,  m4  = 107  and  mg  = 109. 

Then  if  0 £ I < 97-101  = 9797,  s = 2,  N = 5 and  N-s  = 3.  Then  I can  be  re- 
constructed if 

(a)  one  remainder  was  in  error, 
or 

(b)  two  or  one  remainder  are  erased. 

We  now  apply  these  ideas  to  fault  tolerant  computers. 

Let  I]  and  I2  be  two  integers  in  the  range  0 <_  I],  I2  < m = miir^-'-mfl. 
Then  if  I]  and  I2  have  the  remainders 

h - (rll »r12»'  " ,rlN^  * 
and 

12  - (r21 »r22»  — »r2N) » 

then 


^1  t l2^m  = ((rll  1 r2l)mi’  (r12  1 r22)m2’"-’  (rlN  t r2N)mN) 
and 

{I1  • I2)m  ((r11  • P21 • <r12  * r22^m^’ ‘ (rlN  • r2N)mN) 
where  (x)y  means  x mod  y. 

This  result  has  been  previously  suggested  for  use  in  a residue  number 
system  computer.  The  advantage  of  such  a computer  is  that  addition  and 
multiplication  can  be  very  fast.  A disadvantage  is  that  it  is  difficult 
to  compare  the  magnitude  of  I-j  and  I2  from  their  remainders.  We  will  be 
Interested  in  considering  a fault- tolerant  residue  number  system  computer. 
Let  < m^  < •••  < be  pairwise  prime,  and  let  0 < Ii»l2  < M c mim2***m  . 


1 


Consider  the  two  systems  shown  below  where  ‘ ~r 
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is  a box  that  does  addi- 


tion, subtraction  or  multiplication  (perhaps  modulo  some  integer). 


From  the  previous  discussion  we  see  that  the  system  below  the  dotted  line 
will  work  if  T |_£_|  's  produce  faulty  outputs  and  F produce  no  outputs  at  all 
where 

2T  + F £ N-s. 

The  system  above  the  dotted  line  is  the  non-fault-tolerant  version  of  the 
system. 

Note  that  one  can  tolerate  twice  as  many  failed 


+ 

x 


's  as 


's  that 


produce  errors. 
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Thus  one  can  use  an  error  detection  code  to  convert  errors  in  these  boxes 
into  erasures.  One  such  code  that  will  detect  any  single  carry  or  sum 
error  in  an  adder  is  to  multiply  each  remainder  by  3.  Thus  we  would  go 
through  the  following  steps. 

a.  It  - ( r-j i , r12,***,rlN) 
l2  "*  (r21 » r22* ' ‘ ’ »r2N^ 

Encode 

v b.  (rn»  ri2»  * * * — >(  3r-j  i , 3r-|2.  • • • »3r|fl) 

(r21»  r22» * ’ • »r2N)  ^(3r21 » 3r22» • • • »3r2N) 


c.  Form  (3r^  + 3r21 ) 


3m| 


Addition 

+ 


(3r12  + 3^)^  - a2 


(3r1N  + 3r2N)3m  -*•  af 


Actual  results 
may  contain  errors 


d.  Calculate  (a^  (a2)3  •••  (aN)3 

If  («j)  t 0 replace  a^  by  0 (null  symbol), 
alone.  Call  result  (a^ .ag, • • • >aN) 


Otherwise  let 


Decode 


where  a^  = 4 


0 (a*).  f 0 


lT(aV° 


* e.  Reconstruct  + I2  from  (a^ , a2,  •••,aN) 
The  procedure  will  work  if 

(No  of  0’s)  + 2 (No  of  incorrect  a^)  _<  N-s. 
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3.  Coding  for  Noisy  Channels  Including  Multi-User  Channels 

We  have  considered  various  schemes  for  coding  for  multi-user  communica- 
tion channels.  One  such  channel  is  discussed  here— the  modulo  2 channel. 

We  consider  a multiple  access  channel  where  two  users  must  separately 
encode  information  for  a common  channel.  We  assume  word  and  bit  synchroniza- 
tion for  the  encoders  but  that  they  are  unaware  of  the  information  to  be 
transmitted  by  the  other  user.  The  channel  to  be  considered  is  a channel 
which  accepts  a pair  of  binary  symbols  and  produces  as  its  output  a binary 

symbol  which  is  the  modulo  2 summation  of  the  input  symbols. 

Let  R^  and  R2  be  the  rates  of  the  two  users  (in  bits  per  channel  use). 

It  is  well  known  that  the  capacity  region  of  the  modulo  2 channel  is  given 
by  the  equation 

0 < R]  + R2  5.  1 • 

Furthermore  any  point  on  the  line  R-j  + R2  = 1 can  be  achieved  by  time 

sharing  between  two  modes  of  operation  where  in  each  mode  one  encoder  trans- 

mits uncoded  data  and  the  other  encoder  transmits  all  zeros. 

An  alternative  scheme  exists  for  achieving  the  rate  pair 

(Rl ,R2)  = » 1 “ Jr)-  An  (N.k)  binary  cyclic  code  is  chosen  as  the  code 

N 11 

for  encoder  1.  This  code  has  generator  polynomial  g(x)  and  parity  check 
polynomial  h(x).  It  is  assumed  that  N is  an  odd  integer  so  that  g(x)  and 
h(x)  have  no  common  factors. 

Let  encoder  1 transmit  a code  word  from  the  (N,k)  binary  code  and  let 
encoder  2 transmit  a code  word  from  thefi,N-k)  dual  code  with  generator 
polynomial  h(x).  Let  I^(x)  be  the  idempotent  for  the  (N,k)  code  and  let 
Ig(x)  = 1 $ I1 (x)  be  the  idempotent  for  the  dual  code. 

The  decoder  then  receives  a word  of  the  form 
a(x)g(x)  • b(x)  h(x). 

J 


I 
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To  obtain  the  code  word  transmitted  by  encoder  1,  it  multiplies  by  I^(x), 
modulo  X -1.  To  obtain  the  code  word  transmitted  by  encoder  2,  it  multi- 
plies by  I2(x),  modulo  XN-1 . 

This  scheme  is  a special  case  of  the  following:  Encoder  1 transmits 

a word  from  an  (N,k)  binary  code.  A coset  table  is  formed  where  the  coset 
leaders  form  an  (N,N-k)  binary  group  code.  The  receiver  receives  a word  in 
the  coset  table,  say  in  the  i^*1  row  and  column.  It  then  decodes  to  the 
jth  code  word  used  by  encoder  1 and  the  i**1  code  word  used  by  encoder  2. 

The  advantage  of  this  scheme  over  simple  time  sharing  will  be  discussed 
when  we  consider  the  modulo  2 channel  with  errors. 

Modulo  2 Channel  With  Errors 

Let  us  consider  a modulo  2 channel  with  errors  as  the  cascade  of  the 
modulo  2 channel  without  errors  and  a binary  symmetric  channel  with  cross- 
over probability  p.  The  capacity  region  for  this  channel  is  given  by  the 
equation 

0 < Ri  + R2  < 1 - h(p) 
where  h(p)  is  the  entropy  function. 

One  approach  to  coding  for  such  a channel  is  to  time  share  between  two 
modes  of  operation  where  in  one  mode  one  encoder  uses  a t error  correcting 
code  (say  a BCH  code)  while  the  other  encoder  sends  all  zeros.  In  the  other 
mode  the  encoders  switch  roles. 

Another  approach  is  as  follows:  Let  g(x)  be  the  generator  polynomial 

of  a binary  cyclic  code  which  corrects  t errors.  Let  XN-1  = g(x)h-[ (x)h2(x) 
where  N is  odd  so  that  g(x),  h-j (x)  and  h2(x)  have  no  common  factors.  Let 
encoder  1 use  code  words  from  a cyclic  code  with  generator  polynomial 
g(x)h-|(x)  while  encoder  2 uses  code  words  from  a cyclic  code  with  generator 
polynomial  g(x)h2(x). 
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The  received  word  is  of  the  form 

a(x)g(x)h-|  (x)  © b(x)g(x)h2(x)  ©n(x)  = a(x)g(x)  © n(x) 

The  received  word  can  be  decoded  correctly  to  a(x)g(x)  If  no  more  than  t 
errors  occurred  in  n(x).  Then  one  can  find  a(x)  and  b(x)  by  using  the 
idempotents  of  the  codes  with  generators  g(x)h](x)  and  g(x)h2(x). 

The  advantage  of  this  scheme  over  time  sharing  is  that  if  one  source 
is  not  transmitting  (i.e.,  the  encoder  is  transmitting  all  zeros)  the  error 
correction  capability  of  the  code  increases.  For  example,  let  N = 63, 
g(x)  = m-|(x)m3(x)m5(x)m7(x),  h-j(x)  = mg(x)m-|-|  (x)m13(x)  and  h2(x)  = m15(x) 
m23(x)m27(x)m3^ (x)  where  m^(x)  is  the  minimum  function  of  a"*  and  a is  a 
primitive  element  of  GF (64) . Then  g(x)  is  the  generator  polynomial  of  a 
7 error  correcting  code,  g(x)h2(x)  is  the  generator  polynomial  of  an  8 error 
correcting  code.  Thus  if  both  sources  are  transmitting,  4 errors  can  be 
corrected  while  if  only  one  source  is  transmitting  the  code  can  correct  7 
or  8 errors. 
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